Reinforcement Learning for Problems with Hidden State
نویسنده
چکیده
In this paper, we describe how techniques from reinforcement learning might be used to approach the problem of acting under uncertainty. We start by introducing the theory of partially observable Markov decision processes (POMDPs) to describe what we call hidden state problems. After a brief review of other POMDP solution techniques, we motivate reinforcement learning by considering an agent with no previous knowledge of the environment model. We describe two major groups of reinforcement learning techniques: those the learn a value function over states of world, and those that search in the space of policies directly. Finally, we discuss the general problems with these methods, and suggest promising avenues for future research.
منابع مشابه
Using Markov-k Memory for Problems with Hidden-State
TRACA (Temporal Reinforcement learning and Classification Architecture) is a new learning system developed for robot-navigation problems. One difficulty in this area is dealing with problems which contain hidden-state. TRACA solves hidden-state tasks using a novel method to build Markov-k memory chains where k may be of arbitrary length. The seemingly simple strategy employed by TRACA presents ...
متن کاملReinforcement Learning State Estimator
In this study, we propose a novel use of reinforcement learning for estimating hidden variables and parameters of nonlinear dynamical systems. A critical issue in hidden-state estimation is that we cannot directly observe estimation errors. However, by defining errors of observable variables as a delayed penalty, we can apply a reinforcement learning frame-work to state estimation problems. Spe...
متن کاملInstance-based State Identiication for Reinforcement Learning
This paper presents instance-based state identiication, an approach to reinforcement learning and hidden state that builds disambiguat-ing amounts of short-term memory on-line, and also learns with an order of magnitude fewer training steps than several previous approaches. Inspired by a key similaritybetween learning with hidden state and learning in continuous geometrical spaces, this approac...
متن کاملFirst Results with Instance - Based StateIdenti cation for Reinforcement
When a reinforcement learning agent's next course of action depends on information that is hidden from the sensors because of problems such as occlusion, restricted range, bounded eld of view and limited attention, we say the agent suuers from the hidden state problem. State identiication techniques use history information to uncover hidden state. Previous approaches to encoding history include...
متن کاملEvolutionary Search, Stochastic Policies with Memory, and Reinforcement Learning with Hidden State
Reinforcement Learning (RL) problems with hidden state present signiicant obstacles to prevailing RL methods. In this paper , we present experiments conducted using a straightforward approach to solving such problems that trains artiicial neural networks with recurrent connections to represent action policies using evolutionary search. We apply this method to two diierent benchmark problems. Th...
متن کامل